1. Field of the Invention
This invention relates to controlling the types of frame that are used in video compression.
2. Description of the Related Art
A known motion picture encoding system is that known as MPEG (Moving Pictures Expert Group) in which a GOP (Group of Pictures) structure is constructed by a plurality of pictures, each of which is in a data form. In the MPEG structure, an I picture, a P picture and a B picture are specified as picture coding types.
The I picture is an Intra-encoded Picture and is used to denote an intra-frame encoded image. The I picture represents the whole picture plane which is Intra-encoded and is encoded in accordance with the same order as that of the original image. The P picture is a Predictive-Picture and is a predicted encoded frame looking in an inter-frame forward direction, i.e. it is an image which is motion compensated predicted from a past image. In some circumstances the P picture includes an intra-encoded portion of a small block unit in the picture plane and it is encoded in accordance with the same order as that of the original image. A Bi-directionally Predictive-Picture (B) is a bi-directional predictive encoded message, i.e. it is an image which is motion compensation predicted from past and future images. The B picture may include the intra-encoded portion of the small block unit in the picture frame. In a GOP there is at least one I picture and in, for example, MPEG 1 it is specified that the beginning of the GOP is the I picture in a bit stream and the last of the GOP in the order of the original picture plane is an I or P picture.
The bi-directionally predicted B frames improve coding efficiency for a number of reasons:    (a) There are more coding modes to choose from in B frames because B frames can take predictions from past as well as future frames.    (b) It is possible to make predictions of uncovered areas from future frames in B frames.    (c) The possibility of combined predictions, i.e. taking the “average” of two predictions, one from a past and one from a future frame, helps with certain types of motion and with noisy images.
However, using B frames also introduces a number of disadvantages.    (i) In order to be able to predict from future frames, these future frames need to be predicted in the first instance. However, since these predictions have to be made across more frame periods, the frames are usually less correlated and predictions are less accurate. For the sake of clarity, in this disclosure each TV frame is composed of two, normally interlaced, TV fields.    (ii) An object moving at a certain speed might be within the search range if predictions are made from one frame to the next but it might be outside the search range if predictions need to be made two or more frames into the future.
FIG. 1 shows in schematic form how B frame predictions are made from both past and future frames. It also shows that the prediction of P frames has to be made across all B frames between an I or P and a P frame of concern. This rate of difference in time between I and P frames may break the search range of the motion estimation thus reducing picture quality. In practice, in dependence upon the compression algorithm, search range etc., the majority of picture material improves with B frames, but there is always a fraction of sequences which are better encoded without B frames. To avoid the disadvantages of B frames while making use of their advantages, where appropriate, would require adaptive switching between coding with B frames and coding without B frames.
FIG. 2 shows in schematic form an example of an adaptive coding sequence. In FIG. 2 the first part of the coding sequence uses B frames but in the second part B frames are not used. Sequences such as the one shown in FIG. 2 in which B frames are sometimes used and sometimes not used are available in some encoders such as non-real time encoders in which the two parts of the sequence are encoded separately. Therefore the process of adaptively switching B frames on and off is not, of itself, novel. The difficulty however is to decide when B frames should be used and when they should not be used. One arrangement envisaged by the present inventors is to encode two streams in parallel, one with B frames and one without. However, such an arrangement would be very costly for a real time encoder because the encoder would need twice the processing power.
Adaptive and selective coding of video digital signals is disclosed in U.S. Pat. Nos. 5,227,878 and 5,828,787.